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Abstract 

It is possible to derive the maximum entropy principle from ther- 
modynamic stability requirements. Using as a starting point the equi- 
librium probability distribution, currently used in non-extensive ther- 
mostatistics, it turns out that the relevant entropy function is Renyi's 
a-entropy, and not Tsallis' entropy. 



1 Introduction 

The maximum-entropy principle has played an essential role in the devel- 
opment of non-extensive thermostatistics, starting with Tsallis' paper pQ 
in 1988. The Tsallis formalism has known several major revisions without 
reaching a stable and satisfactory status. Origin of the difficulties might be 
that the maximum-entropy principle is used as a postulate of statistical me- 
chanics, while it is rather a result that can be derived from first principles. 
Below is used that the maximum-entropy principle is a consequence of the 
statement that free energy is minimal at equilibrium. The definition of what 
equilibrium is, what free energy is, comes first. The maximum-entropy prin- 
ciple follows. This point of view differs from that of Tsallis literature where 
equilibrium is by definition the state maximizing some entropy functional 
under certain constraints. 

This change of viewpoint has some important consequences. Given a 
family of temperature dependent probability distributions one can calculate 
the entropy functional which is needed to reproduce the given probability 
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distributions. In the present paper the probability distributions used in re- 
cent Tsalllis literature are taken as starting point. Surprisingly, the resulting 
entropy functional is not Tsallis entropy but is the a-entropy of Renyi. This 
result agrees with the known fact [2] that the maximum entropy principle 
in combination with Renyi's entropy reproduces the equilibrium probability 
distributions of non-extensive thermostatistics. Note that there exists an 
extensive literature on Renyi's a-entropies. Some of the references in the 
context of non-extensive thermostatistic are IU IHj • 

The next section serves to fix notations and starts with the probability 
distributions that are found as equilibrium states in recent Tsallis literature. 
Section 3 introduces thermodynamic entropy S(U) as a function of average 
energy U. It is obtained from average energy by integrating the well-known 
thermodynamic definition of temperature. In Section 4 the requirement of 
thermodynamic stability is worked out. In Section 5 the maximum entropy 
principle is derived. The final section contains a short discussion of results. 
Some of the calculations are given in Appendix. 



2 Equilibrium probability distributions 

For sake of simplicity notations refer to a classical gas of n particles enclosed 
in a container of volume V in //-dimensional space. The energy functional is 
denoted H(x), with x a point of phase space T. 
Average energy 



U(f3) = j dxp p (x)H(x) (1) 
is calculated using the distribution j^l E] 

Vf>(?) = [1 " (1 " 9)P[H(x) ~ Urn? 1 "* ■ ( 2 ) 

In this expression Z is the normalization 

Z(f3) = J dx [1 - (1 - q)P[H{x) - U{(3)]]f l ~ q) ■ (3) 
The notation [u]+ = max{0,M} is used. Equation (j2J can be written as 

Pfi(x) = [1 " (1 " q)FH(xW^-0 (4) 



with 



^ l + (l-q)/3U(py (5) 
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In the latter form normalization is straightforward and average energy can 
be calculated 

U{(5) = J dx [1 - (1 - q)PH(x)))W-ti H (x) (6) 

The pair of equations (0113) should have a unique solution U{0). In addition, 
this solution should be a decreasing function of (3. See Appendix A for a 
discussion of the existence of solutions. 



3 Thermodynamic entropy 

The basic thermodynamic quantity is entropy S(U) as a function of energy 
U . It is used to define inverse temperature (3 by 

In the present context U(0) is given. Hence S(U) can be obtained by a 
simple integration of (J7J). This is done in Appendix B. The result is 



" / TYl \ vn 



Here, m is the mass of each particle, and h is a constant inserted for dimen- 
sional reasons (units are used in which ks = 1). The integration constant 
has been chosen so that for the ideal gas in the limit q — 1 one obtains the 
standard result 

!(t) = =b^-=b=. (9) 
4 Thermodynamic stability 

Thermodynamic stability of equilibrium states requires jH] in the first place 
that 

This condition boils down to the requirement that U{(3) is a decreasing func- 
tion of p. This is assumed to be the case, but cannot be proved under general 
conditions for the family of probability distributions (0). 
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If firU|) is satisfied for all U then S(U) can be replaced by its Legendre 
transform, which is called Helmholtz free energy F, and which historically is 
defined by 

F = U-TS(U) (11) 

with temperature T given by T = 1//3. Free energy F is a concave function 
of temperature T. Its first derivative satisfies 

Thermodynamic stability is more than just (|10|). The system should be 
stable against all possible perturbations. For that reason the free energy 
should be concave, not only as a function of temperature but in a much 
larger parameter space. To formulate this requirement in a consistent way it 
is easier to work with f3F = (3U — S instead of with F. Note that 

±jf>F = U. (13) 

Hence the condition of concavity of F(T) is equivalent with concavity of /3F 
as a function of (3. The quantity (3F depends on both (3 and on the choice 
of energy function H(x), but only the combination (3H(x) occurs. Hence 
the requirement that /3F is concave in (3 can be extended to the requirement 
that f3F is a concave function over the space of all possible (3H(x). The 
latter requirement is too strong, because of its global nature, and should be 
weakened by requiring local concavity of j3F. A tangent plane to (3F in the 
point (3H is given by the linear map 

(3'H' f3F + (3'(H') -(3(H) (14) 

where averages are equilibrium averages w.r.t. (3, H. The requirement for 
stability of the system with energy functional H(x) at inverse temperature 
(3 is therefore that 

[3'F' < (3F + f3'(H') - (3(H) (15) 

for all possible choices of (3',H'. Note that ()15j) can be written as 

S' - (3'U' >S - (3(H'). (16) 

In this expression, the averages (•) are still calculated w.r.t. the equilibrium 
probability distribution of the system with energy functional H(x) at inverse 
temperature (3. 
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5 Maximum entropy principle 

The origin of the maximum entropy principle lies in the requirement of ther- 
modynamic stability. Indeed, (fT7)|) can be read as the statement that the 
quantity S — f3(H') is maximal when (3 = f3' and H = H', i.e., when for the 
calculation of free energy the correct probability distribution is used. This 
is the maximum entropy principle. In the canonical formalism of statistical 
physics, based on the Boltzmann-Gibbs distribution, systems with a finite 
number of degrees of freedom are generically stable. Hence it is acceptable 
to interchange primed and unprimed systems. In the more general context 
of non-extensive thermostatistics this interchange is not acceptable, since it 
is known (see e.g. P) that instabilities can occur. One concludes that the 
maximum entropy principle holds only under the condition that the varia- 
tion of S — f3(H') is restricted to those pairs of inverse temperature (3 and 
energy functional H(x) which are thermodynamically stable. Of course, this 
statement is of an academic rather than practical value. But it shows that 
maximum entropy should not be introduced in an axiomatic manner. It also 
gives somewhat more freedom in deriving a variational principle which is 
useful for practical purposes because it is not necessary to prove that the 
variational solution corresponds with a maximum. Indeed, in principle the 
variation should be restricted to the equilibrium distributions of stable sys- 
tems, and by definition of stability these will have a lower free energy than 
the variational solution. 

Let us now introduce an entropy functional which reproduces (J2J) by 
means of the maximum entropy principle. Inspection of (JHJ) learns that the 
correct ansatz for entropy as a function of probability distribution is 



This expression is proportional to Renyi's a-entropy with a = 1/q. One veri- 
fies immediately, using the expressions from Section 2, that I{pp) = S(U((3)). 
Variation of 




(17) 



£(p) = /(p)-/3 



r Ur 



dxp{x)[H{x) -U{(3)]-a [dxp(x)-l (18) 



gives 



p(x) 



1 



(1 



q)a - (1 



q)P[H(x) - U<M 



9/(1-9) 



(19) 



Z(P) 



+ 



with 




(20) 
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Integrating p(x) 1 ^, as given by (fl^jl. gives 



Z(P) = J dx [-(1 - q)a - (1 - gMF(z) - 

= -(1 - q)a j dx [-(1 - q)a - (1 - q)P[H(x) - U((3)]]f x ' q) . 

(21) 

Hence a = — 1/(1 — q) results in a correctly normalized probability distribu- 
tion. This shows that (fTH|) coincides with (J2J). One concludes that there exists 
a maximum entropy principle which produces (j2J) as the equilibrium proba- 
bility distributions and is such that the attained maximum value coincides 
with thermodynamic entropy. 



6 Discussion 

This paper contains one message and one result. The message is that the 
maximum entropy principle is a consequence of the requirement of thermo- 
dynamic stability. In the context of the Boltzmann-Gibbs distribution this is 
an evidence. In that case systems with a finite number of degrees of freedom 
are automatically stable. This allowed Jaynes ^0] to attribute an axiomatic 
status to the maximum entropy principle. In the current version of Tsallis 
thermostatistics systems with a finite number of degrees of freedom are not 
automatically stable. The conclusion of the present paper is therefore that 
maximum entropy should not be used in an axiomatic manner, but should 
be derived from the requirement of thermodynamic stability. 

The result of the present paper is that the equilibrium probability dis- 
tributions, as found in recent Tsallis literature, can be derived by means of 
the maximum entropy principle. However, the relevant entropy functional is 
Renyi's a-entropy rather than Tsallis' entropy functional. Of course, there 
is an easy connection between the two entropies [TT]. But the present paper 
shows that a thermodynamically correct definition of temperature leaves no 
choice but to use Renyi's entropy instead of that of Tsallis. 

Mainstream non-extensive thermostatistics is based on a variational prin- 
ciple which involves escort probability distributions |SlCj- The main objec- 
tion, from the present point of view, against these papers is that they do 
not reproduce thermodynamic entropy ©• As a consequence, the main- 
stream derivation of (0) has problems with a correct definition of temper- 
ature. Bashkirov, in the conclusions of his paper hopes that one "can 
abandon the use of escort probabilities for calculations of average values in 
favor of original distributions." This statement seems to be exaggerated. We 
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know that escort probabilities are important in fractal physics ^2]- As shown 
in E2] they also play an important role in generalized thermostatis- 

tics. It only happens that these escort probabilities are not involved in the 
formulation of the maximum entropy principle. 

Appendix A 

In this appendix the solution of ©EU) is discussed. Elimination of U{(3) gives 

= \ + jf dx [1 - (1 - q)f3*H{x)]]f ^ (1 - (*) 

y dx [1 - (1 - . (Al) 
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/3 /3* P*Z*(J3) 
This simplifies to 

f r dx [1 - (1 - g)/r#(x)]]i /M 

13* = p& — V ;JJ + r . A2 

J r dx [1 - (1 - q)(3*H(x)]]f l - q) 

See ^E] for methods to prove the existence and uniqueness of the solution f3* 
of this equation. Then average energy is given by 

U(f3) = z^— (lz ~ 4V (A3) 



1 -q \(3* (3 

Thermodynamic stability requires that U(j3) is a decreasing function of (3. 
There is no general argument why this should always be the case. 



Appendix B 

In this appendix entropy S{U) is calculated. One has 
= {l-q)l3[{H)-U{i3)} 

= J dx [1 - (1 - q)/3[H(x) - U(f3)}] q / [1 - q) (1 - ?)/9[#(s) - U((3)} 

l — [ dx [1 - (1 - q)(3[H{x) - U({3)fl {l ~ q) + 1- (Bl) 



Z((3) Jr 
Hence one obtains 



Z{0) = / dx [1 - (1 - q)/3[H(x) - ■ (B2) 



From the latter expression follows 




(1 - q)P[H(x) - U{/3)]] 



9/(1-9) 



+ 



x H(x)-U((3) 





(B3) 



In the last line (J7J) has been used. Integration gives 



S(U(/3)) = \nZ(/3) + constant. 



(B4) 



This is (|SJ), up to the integration constant. 

Let us now show that (JHJ) in case of the ideal gas and in the limit q = 1 
reduces to (jUJ). For the ideal gas, in the limit q = 1, is 



This yields © because (3U((3) = un/2. 
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